Design and evaluation of an interactive quality dashboard for national clinical audit data: a realist evaluation

BackgroundNational audits aim to reduce variations in quality by stimulating quality improvement. However, varying provider engagement with audit data means that this is not being realised.AimThe aim of the study was to develop and evaluate a quality dashboard (i.e. QualDash) to support clinical teams’ and managers’ use of national audit data.DesignThe study was a realist evaluation and biography of artefacts study.SettingThe study involved five NHS acute trusts.Methods and resultsIn phase 1, we developed a theory of national audits through interviews. Data use was supported by data access, audit staff skilled to produce data visualisations, data timeliness and quality, and the importance of perceived metrics. Data were mainly used by clinical teams. Organisational-level staff questioned the legitimacy of national audits. In phase 2, QualDash was co-designed and the QualDash theory was developed. QualDash provides interactive customisable visualisations to enable the exploration of relationships between variables. Locating QualDash on site servers gave users control of data upload frequency. In phase 3, we developed an adoption strategy through focus groups. ‘Champions’, awareness-raising through e-bulletins and demonstrations, and quick reference tools were agreed. In phase 4, we tested the QualDash theory using a mixed-methods evaluation. Constraints on use were metric configurations that did not match users’ expectations, affecting champions’ willingness to promote QualDash, and limited computing resources. Easy customisability supported use. The greatest use was where data use was previously constrained. In these contexts, report preparation time was reduced and efforts to improve data quality were supported, although the interrupted time series analysis did not show improved data quality. Twenty-three questionnaires were returned, revealing positive perceptions of ease of use and usefulness. In phase 5, the feasibility of conducting a cluster randomised controlled trial of QualDash was assessed. Interviews were undertaken to understand how QualDash could be revised to support a region-wide Gold Command. Requirements included multiple real-time data sources and functionality to help to identify priorities.ConclusionsAudits seeking to widen engagement may find the following strategies beneficial: involving a range of professional groups in choosing metrics;real-time reporting;presenting ‘headline’ metrics important to organisational-level staff;using routinely collected clinical data to populate data fields;and dashboards that help staff to explore and report audit data. Those designing dashboards may find it beneficial to include the following: ‘at a glance’ visualisation of key metrics;visualisations configured in line with existing visualisations that teams use, with clear labelling;functionality that supports the creation of reports and presentations;the ability to explore relationships between variables and drill down to look at subgroups;and low requirements for computing resources. Organisations introducing a dashboard may find the following strategies beneficial: clinical champion to promote use;testing with real data by audit staff;establishing routines for integrating use into work practices;involving audit staff in adoption activities;and allowing customisation.LimitationsThe COVID-19 pandemic stopped phase 4 data collection, limiting our ability to further test and refine the QualDash theory. Questionnaire results should be treated with caution because of the small, possibly biased, sample. Control sites for the interrupted time series analysis were not possible because of research and development delays. One intervention site did not submit data. Limited uptake meant that assessing the impact on more measures was not appropriate.Future workThe extent to which national audit dashboards are used and the strategies national audits use to encourage uptake, a realist review of the impact of dashboards, and rigorous evaluations of the impact of dashboards and the effectiveness of adoption strategies should be explored.Study registrationThis study is registered as ISRCTN18289782.FundingThis project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in full in Health and Social Care Delivery Research;Vol. 10, No. 12. See the NIHR Journals Library website for further project information.

In the eye of the storm: impact of COVID-19 pandemic on admission patterns to paediatric intensive care units in the UK and Eire.

BACKGROUND: The coronavirus disease-19 (COVID-19) pandemic had a relatively minimal direct impact on critical illness in children compared to adults. However, children and paediatric intensive care units (PICUs) were affected indirectly. We analysed the impact of the pandemic on PICU admission patterns and patient characteristics in the UK and Ireland. METHODS: We performed a retrospective cohort study of all admissions to PICUs in children < 18 years during Jan-Dec 2020, using data collected from 32 PICUs via a central database (PICANet). Admission patterns, case-mix, resource use, and outcomes were compared with the four preceding years (2016-2019) based on the date of admission. RESULTS: There were 16,941 admissions in 2020 compared to an annual average of 20,643 (range 20,340-20,868) from 2016 to 2019. During 2020, there was a reduction in all PICU admissions (18%), unplanned admissions (20%), planned admissions (15%), and bed days (25%). There was a 41% reduction in respiratory admissions, and a 60% reduction in children admitted with bronchiolitis but an 84% increase in admissions for diabetic ketoacidosis during 2020 compared to the previous years. There were 420 admissions (2.4%) with either PIMS-TS or COVID-19 during 2020. Age and sex adjusted prevalence of unplanned PICU admission reduced from 79.7 (2016-2019) to 63.1 per 100,000 in 2020. Median probability of death [1.2 (0.5-3.4) vs. 1.2 (0.5-3.4) %], length of stay [2.3 (1.0-5.5) vs. 2.4 (1.0-5.7) days] and mortality rates [3.4 vs. 3.6%, (risk-adjusted OR 1.00 [0.91-1.11, p = 0.93])] were similar between 2016-2019 and 2020. There were 106 fewer in-PICU deaths in 2020 (n = 605) compared with 2016-2019 (n = 711). CONCLUSIONS: The use of a high-quality international database allowed robust comparisons between admission data prior to and during the COVID-19 pandemic. A significant reduction in prevalence of unplanned admissions, respiratory diseases, and fewer child deaths in PICU observed may be related to the targeted COVID-19 public health interventions during the pandemic. However, analysis of wider and longer-term societal impact of the pandemic and public health interventions on physical and mental health of children is required.

Severity of COVID-19 in children with cancer: Report from the United Kingdom Paediatric Coronavirus Cancer Monitoring Project.

BACKGROUND: Children with cancer are frequently immunocompromised. While children are generally thought to be at less risk of severe SARS-CoV-2 infection than adults, comprehensive population-based evidence for the risk in children with cancer is unavailable. We aimed to produce evidence of the incidence and outcomes from SARS-CoV-2 in children with cancer attending all hospitals treating this population across the UK. METHODS: Retrospective and prospective observational study of all children in the UK under 16 diagnosed with cancer through data collection from all hospitals providing cancer care to this population. Eligible patients tested positive for SARS-CoV-2 on reverse transcription polymerase chain reaction (RT-PCR). The primary end-point was death, discharge or end of active care for COVID-19 for those remaining in hospital. RESULTS: Between 12 March 2020 and 31 July 2020, 54 cases were identified: 15 (28%) were asymptomatic, 34 (63%) had mild infections and 5 (10%) moderate, severe or critical infections. No patients died and only three patients required intensive care support due to COVID-19. Estimated incidence of hospital identified SARS-CoV-2 infection in children with cancer under 16 was 3%. CONCLUSIONS: Children with cancer with SARS-CoV-2 infection do not appear at increased risk of severe infection compared to the general paediatric population. This is reassuring and supports the continued delivery of standard treatment.